Simulation framework for connected vehicles: a scoping review

Background: V2V (Vehicle-to-Vehicle) is a booming research field with a diverse set of services and applications. Most researchers rely on vehicular simulation tools to model traffic and road conditions and evaluate the performance of network protocols. We conducted a scoping review to consider simulators that have been reported in the literature based on successful implementation of V2V systems, tutorials, documentation, examples, and/or discussion groups. Methods: Simulators that have limited information were not included. The selected simulators are described individually and compared based on their requirements and features, i.e., origin, traffic model, scalability, and traffic features. This scoping review was reported according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR). The review considered only research published in English (in journals and conference papers) completed after 2015. Further, three reviewers initiated the data extraction phase to retrieve information from the published papers. Results: Most simulators can simulate system behaviour by modelling the events according to pre-defined scenarios. However, the main challenge faced is integrating the three components to simulate a road environment in either microscopic, macroscopic or mesoscopic models. These components include mobility generators, VANET simulators and network simulators. These simulators require the integration and synchronisation of the transportation domain and the communication domain. Simulation modelling can be run using a different types of simulators that are cost-effective and scalable for evaluating the performance of V2V systems in urban environments. In addition, we also considered the ability of the vehicular simulation tools to support wireless sensors. Conclusions: The outcome of this study may reduce the time required for other researchers to work on other applications involving V2V systems and as a reference for the study and development of new traffic simulators.


Introduction
In recent decades, a significant increase in vehicle use has increased traffic congestion and fatalities 1 . According to the World Health Organization, 1.25 million people are killed and severely injured involving vehicle accidents 2 . Hence, connected vehicle technology responds to this constraint, aiming to leverage inter-vehicle communication to produce safe, userfriendly, and fuel-efficient vehicle assistive technologies 3,4 . One of the main aspects of connected vehicle research is to optimise traffic flow through the exchange of information 5 . This communication can be sorted in terms of vehicle (V2V), infrastructure (V2I), a pedestrian (V2P), and network (V2N) 6,7 . The exchange of information, collectively known as V2X communications, could assist drivers in preventing accidents by providing warnings of danger invisible to drivers and other sensors (e.g. collision avoidance, lane departure warning and speed limit alert) 8,9 . Nevertheless, the adoption of connected vehicle technology poses a range of challenges, particularly in urban environments. It is challenging to analyse the effectiveness of the application of connected vehicles under traffic conditions [10][11][12] . As such, simulations using traffic and network simulators as well as mobility generators are viable alternatives to modelling and determining the effectiveness of such deployments in the real world 13,14 , as it provides an affordable and scalable method for analysing model compliance in various contexts and parameters. Traffic simulations are categorised by level of detail into three separate categories 15 . First, the most precise information on each vehicle in the system is microscopic simulations 16 . Second, mesoscopic simulations exploit aggregate velocity-density functions to represent their behaviour and view traffic as a continuous stream of vehicles 17 . Finally, macroscopic simulation is the large-scale traffic model, which focuses on combined traffic status 18 . Microscopic simulations provide the highest degree of detail for modelling, although they are the slowest to execute 19,20 .
In addition, mobility generators are a possible option for modelling vehicle elements such as traffic, temporal and spatial mobility, and generating mobility traces 21,22 . These traces are then uploaded to a network simulator, which mimics vehicle-to-vehicle communication. Furthermore, these traces can be generated by observing real-world vehicles on the road and then used in network simulations 23,24 . The effect of network parameter modifications on traffic mobility is a strategic objective simulation 25 . It is also restricted to the use of the trace controlled by the mobility model. Another option is to use a simulator that directly integrates the mobility framework.
This study focuses on the Vehicular Adhoc Networks (VANET) which relies on network protocols to assess their performance 22,23 , given that actual experiments are not possible. Over the last decade, efforts have been made to produce a full transport simulator for VANET solutions, including a wireless network simulator for modelling and evaluation 24,25 . A wide range of simulators can be used for VANET simulation modelling, both commercial and open source. Older simulators provide a network simulator to communicate with stationary mobility models. Many researchers have examined various mobility models with simulation tools for several contexts. Such simulator tools are not yet well explored since many researchers base their simulations depending on their use case settings. Thus, this motivates the identification of different simulators do not yet exist. Therefore, this study conducted a systematic scoping review to identify the applicability and availability of existing mobility generators, network simulators, and combination simulators.
The rest of the paper is organised as follows. The following section explains the methods used for analysing existing mobility generators and network simulators. Further, the outcome of the review scoping is depicted with several discussions. The final section concludes by considering future directions.

Methods
A This work involves identifying the research question, identifying relevant studies, selecting the studies, charting the data, collating, summarising, and reporting results. The review was carried out in compliance with the PRISMA Extension for Scoping Review 26 .

Inclusion criteria
Under standard procedures for performing scoping reviews 26 , published primary studies on VANET were considered for inclusion. Although research may be conducted in any country without focusing on language restrictions, we only obtained data from studies published in English-language journals. Studies had to include mobility generators, network simulators, and vehicle network simulators. We included studies related to vehicular communications that investigated V2V safety

Amendments from Version 1
The abstract and introduction were improved to highlight the focus of the study which contributes to the domain of Vehicular Adhoc Networks (VANET) by describing the mobility generators and network simulators suitable to be considered in the context of connected vehicles or V2V. The sections organization was added in the last paragraph of the Introduction section. The Method section has been re-written to provide better presentation and clarity. The Result section was also improved to describe the findings of the scoping review in a clearer way. Similarly, the Discussion section also has been improved, highlighting the contribution and benefit of this study. The conclusion includes the recommendation of simulators for realtime modelling and recommendation for extending this study.
The citations and references in the reference list are unified according to a Harvard referencing style, and the paper was proofread and edited accordingly.

Any further responses from the reviewers can be found at the end of the article
applications, vehicle network performance, driver behaviour, and vehicle simulation tools. We excluded studies prior to the year 2015.
Databases IEEE Xplore and Science Direct were used to perform in-depth searches of the information included in these databases.

Literature Search Strategy
The search method included controlled vocabulary and free-text word phrases generally linked to (1) network simulation, mobility generators, or network simulators, and (2) VANET, vehicle, or nodes. All searches were initiated in November 2019 and December 2020, when the database was updated. Endnote was used to import the search results 27 .

Citation Screening
After removing duplicate studies, search results were exported for screening. This was done to filter references based on the above-mentioned inclusion criteria. To minimise the possibility of bias, each reference was checked twice by two team members, and the team addressed any inconsistencies. The first screening process ended in April 2020, and the outcomes were updated in January 2021.

Data Extraction
To extract data from each included research, a spreadsheet was created using Microsoft Excel. Several rounds of piloting the data extraction spreadsheet was performed, during which all team members collected data from the same research, and the results were reviewed during team meetings to ascertain content consistency. The piloting extraction process guaranteed that all relevant data fields were collected and that the content was uniform across the research team. After familiarising all team members with the data extraction method, studies were allocated to each member, and the relevant data were extracted separately. Year, country, mobility generators, network simulators, active development, release date, licence, predefined map, traffic model, architecture language, and simulation language were collected from each included research where accessible. In March 2021, data extraction was finalised.

Data Analysis
The objective outcomes of all included studies were retrieved in the order in which they were reported. The research team then classified these findings according to their similarity to the measured concepts: routing protocol, scenario, mobility generator, and network simulator. Subjective outcomes were similarly retrieved in the way described in the publications and then classified according to their similarity to the assessed concepts: contribution. The subjective outcome criteria for each study were extracted and operationalised by consensus among our research team.

Results
The initial search turned up 269 matches. After removing duplicates, a total of 184 titles and abstracts were screened, from which 72 publications were subjected to full-text review. 10 studies fulfilled the criteria for inclusion and were included in the analysis (see Figure 1 for the PRISMA Flow Diagram).  We found that open-source mobility and network simulators were popular among researchers. Microscopic models were preferable for research related to vehicular communications since the simulations provide the most precise information of each vehicle or mobile node and the highest degree of detail for modelling compared to macroscopic and mesoscopic models. Common network simulators were NS-2, Ns-3 and OMNeT++. However, not all mobility simulators supported active development, which is important in current active research domains such as vehicular communications. The mobility generators and simulators available after 2015 are further shown in Table 1 and  Table 2, respectively. Besides, summarised previous studies are shown in Table 3 of using mobility generators or simulators.

Discussion
Since this area of study is considered as a relatively new but rapidly growing field, this scoping review process only considers relevant papers published from 2015 onwards, which shows that extensive research has been conducted to create security standards for communication technologies, particularly the vehicular network. Although various simulators can be enhanced with library extensions, none of the simulators is related to security and privacy. Ultimately, researchers and professionals cannot compare their security measures to a given circumstance. For instance, ensuring the privacy of a vehicular user in a fast-moving network and disseminating messages in a secure vehicular environment. However, there is no simple practice of extending existing simulators to the desired security standard, which implies that future development research will need to be done.
In addition, the quality of a simulation depends largely on the precision of the models. The range of precision has increased dramatically recently, where several modules contain signal attenuation components, multiple antenna models, and environmental interferences. However, one continuous barrier to producing accurate simulations is the evolution of rapid prototyping and its increasing use in-vehicle networks. For example, vehicle nodes would depend on three-dimensional scenarios to communicate with other nodes. It would be crucial for current and future simulators to extend the current simulators to these new conditions.  This paper uncovers an automatic routing protocol for the VANET scenario. The idea is to disseminate the information provided by several roadside units. There are three routing protocols evaluated using several performance metrics in terms of delay, number of hops, total service time, and number of fragments.
The paper focuses on two routing protocols within the VANET scenario. The idea is to ensure an optimal path from source to destination under a few performance measures in terms of throughput and packet delivery ratio.  Apart from that, integration with real-time system modelling based on non-real-time events creates additional challenges. Due to resource limitations, current simulators do not correspond with the physical properties of the hardware prototype while simulating a comprehensive network with multiple vehicles. Several alternatives have been put forward to reduce the complexity that could speed the simulation. However, this approach usually does not include indirect outcomes, which could seriously impact the behavior of real-world network components. It is, therefore, necessary to examine the interconnection between simulators and hardware devices with the security standards concerned.

Conclusions
Studies have led to the discovery of comprehensive and realistic simulation tools due to the increasing popularity and interest for the future transportation system. This work has examined the current availability of simulators. While testing VANET with essential performance, it is necessary to deploy a mobility generator and mobility network that accurately represent real vehicle traffic. Based on our comparative identification, NS-3 and SUMO has been the optimal choice for real-time VANET modelling. Although several simulators have many features, it is worth exploring further the improvement of the simulators for specific scenarios. In addition, this work can be further be expanded in future by investigating relationships of appropriate simulator for a V2V or V2X application to different scenarios and protocols. We plan to study the used simulators in this context and the extent of benefit and development achieved using the simulators.

Data availability
Underlying data All data underlying the results are available as part of the article and no additional source data are required.

Open Peer Review
Communication, Sensors and systems.
I confirm that I have read this submission and believe that I have an appropriate level of expertise to confirm that it is of an acceptable scientific standard, however I have significant reservations, as outlined above.
The benefits of publishing with F1000Research: Your article is published within days, with no editorial bias • You can publish traditional articles, null/negative results, case reports, data notes and more • The peer review process is transparent and collaborative • Your article is indexed in PubMed after passing peer review • Dedicated customer support at every stage • For pre-submission enquiries, contact research@f1000.com